Unbiased cut selection for optimal upper 
limits in neutrino detectors: the model 
rejection potential technique 

Gary C. Hill* and Katherine Rawlins^ 

Department of Physics 
University of Wisconsin, Madison 
1150 University Avenue 
Madison WI 53706 USA 

Abstract 

We present a method for optimising experimental cuts in order to place the strongest 
constraints (upper limits) on theoretical signal models. The method relies only on 
signal and background expectations derived from Monte-Carlo simulations, so no 
bias is introduced by looking at actual data, for instance by setting a limit based 
on expected signal above the "last remaining data event." After discussing the 
concept of the "average upper limit," based on the expectation from an ensemble 
of repeated experiments with no true signal, we show how the best model rejection 
potential is achieved by optimising the cuts to minimise the ratio of this "average 
upper limit" to the expected signal from the model. As an example, we use this 
technique to determine the limit sensitivity of kilometre scale neutrino detectors to 
extra-terrestrial neutrino fluxes from a variety of models, e.g. active galaxies and 
gamma-ray bursts. We suggest that these model rejection potential optimised limits 
be used as a standard method of comparing the sensitivity of proposed neutrino 
detectors. 
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1 Introduction 



In this paper, we examine the problem of choosing experimental cuts in order 
to place the most restrictive limits on theoretical signal models. How should 
one choose cuts with this in mind? If a signal is assumed, the goal is to max- 
imise the significance of the observation, by for instance optimising signal to 
noise or signal to square root noise. If, however, one assumes that no signal will 
be observed, a different technique is required to optimise the "model rejection 
potential," i.e. the limit setting potential of an experiment. This technique 
must not depend on the experimental data, since choosing cuts based on the 
data (e.g. cutting after the last remaining event) leads to confidence intervals 
that do not have frequentist coverage. 

In this paper, we describe and assess an unbiased method [1,2], based only on 
Monte Carlo signal and background expectations. 

Firstly we discuss how an experimental observation is used to set a limit on 
a theoretical flux (section 2). The desire to minimise the upper limit (and 
thus place the strongest constraint on the theoretical model) leads to the 
concepts of "average upper limits" and "model rejection factor," which are 
discussed in section 3. We show how choosing cuts based on optimising the 
"model rejection factor" leads to, on average, the best possible limit from 
the experiment. In section 4, we illustrate this technique by calculating the 
model rejection potential of a kilometre scale neutrino detector with respect to 
predicted extra-terrestial neutrino fluxes such as active galaxies and gamma- 
ray bursts. 



2 Calculating flux limits 

When an experiment fails to detect an expected flux, an upper limit on that 
flux can be derived from the experimental observation. As an example consider 
a detector that should observe 100 signal events above an expected background 
of a few events. If only the few events consistent with background are observed, 
then obviously the model is strongly constrained as the 100 expected events 
were not seen. The exact calculation of an upper limit on the source flux would 
proceed as follows. The theoretical source flux, ^[E,6), is convolved with the 
detector response and after some cuts yields an event expectation, Ug. Let the 
expected background for the same analysis and cuts be rib. The experiment is 
performed and Uobs events are seen. 
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In this experiment the 90% confidence^ interval yUgo = {fii, ^12) is a function 
of the number of observed events, Uobs, and of the expected background Uf, 



fJ'9o{n obs,nb) 



(1) 



For example, an experiment that observes 3 events on an expected background 
of 1.5 would report a 90% confidence interval in the Feldman- Cousins [4] "uni- 
fied" approach of yU9o(3, 1.5) = (0.0,5.92). Note that this confidence interval 
includes 0.0, and therefore /i2 = 5.92 is an upper limit. In what follows we 
shall take /igo to mean this upper limit. 

The corresponding upper limit on the source spectrum ^{E, 9) is found by 
scaling the source fiux by the ratio of the upper limit to the signal expecta- 
tion [7] 



If the expected signal contribution to the observation was Ug = 100, then the 
90% confidence level upper limit on the source fiux is 



In this case the theoretical model predicts a large number of events with very 
few seen and is therefore severely constrained by the experiment. A low value of 
the ratio /i9o(^o6s, nf,)/ns has led to a strong constaint on the model, therefore 
one should optimise cuts to make this ratio small. However, the observed upper 
limit depends on Uobs which is not known until the cuts are defined and the 
experiment performed! Fortunately, the classical concept of an ensemble of 
experiments allows us to calculate an "average upper limit" (equivalent to the 
Feldman- Cousins "sensitivity" [4]) which can take the place of fJ^9o{nobs,nb) in 
the signal to upper limit calculation. 



^ For simplicity we will only use 90% confidence intervals in this discussion; the 
results of course are applicable to any level of confidence or any formulation of 
confidence intervals - whether frequentist or Bayesian. 





n 



'S 




(3) 
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3 Average upper limits and "model rejection factor" 



Although we cannot know the actual upper limit that will result from an ex- 
periment until looking at the data, we can use the Monte Carlo predictions 
to calculate the average upper limit (Feldman-Cousins "sensitivity" [4]) that 
would be observed after hypothetical repetition of the experiment with ex- 
pected background and no true signal {us = 0). In the example of the 
previous section, the expected background was rib = 1-5. Over an ensemble of 
experiments with no true signal, this background will fluctuate to Uobs — 
with Poisson probability 22.3% and an upper hmit of /i9o(0, 1.5) = 1.33 re- 
ported. The background will fluctuate to Uobs = 1 in 33.5% of experiments, 
where an upper limit of 2.91 would be reported, and so on for each value of 
Hobs- The "average upper limit" is the sum of these expected upper limits, 
weighted by their Poisson probability of occurrence 

°° (nh)"'°'"' 
/igoK) = J2 l^9o(nobs,nb) -, r|-exp(-nb) (4) 

no6.=0 \Pohs)- 



This average upper limit is shown in figure 1 as a function of the expected 
background for four different Poisson confidence levels under the Feldman- 
Cousins "unified" ordering principle. Before performing the experiment, one 
can see what background would be expected to remain as a function of various 
cuts, then consult this plot to find out what average upper limit result the 
experiment would be expected to give. 

Over an ensemble of identical experiments, the strongest constraint on the 
expected signal flux ^{E, 9) corresponds to the set of cuts that minimises the 
"model rejection factor" 



and hence minimises the average flux upper limit that would be obtained over 
the hypothetical experimental ensemble 

^E,eu = m.e)^ (6) 

Us 

In the actual experiment we will not obtain $(£', 6*) go, but obtain one limit 
based on the observed number of background counts, which depends on how 
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Fig. 1. Average 90% Feldman-Cousins upper limit that would be obtained by an 
ensemble of experiments with no true signal in the presence of expected background. 

the background happened to fluctuate in the single, real experiment. However, 
prior to performing the experiment, the average flux upper limit (flux times 
model rejection factor) tells us what wc would expect over repeated runs of 
the real experiment, and on average the best limit will come from choosing 
cuts that minimise this average limit. Also, the model rejection factor curve 
tells us how sensitive the expected hmit is to the choice of cuts. 

A somewhat common practice is to choose the flnal cut based on the last 
observed data event. The event limit /xgo then comes from the observation 
of zero events above that cut. This event limit is then compared to the pre- 
dicted signal from the source flux above the cut, and the flux limit obtained 
using equation 2. Calculating the event limit in this way results in confldence 
intervals which do not have frequentist coverage. The classical concept of fre- 
quentist coverage [3,4,5] involves the construction of (100 x a)% confldence 
intervals, such that in (100 x «)% of repeated experiments, the reported con- 
fldence interval will include ( "cover" ) the flxed but unknown true value of the 



5 



estimated parameter /x^. This coverage applies to any value that /Xt may take. 
Choosing the cut based on the last observed data event will lead to intervals 
that do not cover all possible values of with the correct frequency, as can 
be seen by a simple example. Assume that we are using a standard Neyman 
construction for a Poissonian upper limit. As a example, the 90% confidence 
level upper limit for zero observed events is = 2.3. If we always choose the 
cut such that zero events are observed, then we will always report an upper 
limit n = 2.3. Thus, in any experiments where the true value of the parameter 
II is greater than 2.3, the reported confidence interval, (always /i < 2.3), never 
covers the true value! At the other extreme, for true values /it < 2.3, the re- 
ported confidence interval will cover the true value in exactly 100% of cases. In 
neither case is the required 90% coverage achieved, showing that choosing the 
final cut based on the observed data is not a valid method in the frequentist 
framework. 

These considerations show that the final experimental cuts must be chosen 
prior to looking at the data, and that the model rejection potential technique 
provides a method of optimising these cuts to give the best average limit. 
In the next section we apply the model rejection potential technique to the 
determination of the sensitivity of proposed kilometre scale neutrino detectors 
to astrophysical sources of neutrinos, using searches for both diffuse and point 
like emissions as examples. 



4 Model rejection potential of kilometre scale neutrino detectors 

Various proposals exist for the construction of kilometre scale neutrino de- 
tectors [6] in the the deep ocean (the ANTARES, NESTOR and NEMO ex- 
periments) or in Antarctic ice (the Icecube experiment). These experiments 
look for upward moving muons which result from neutrino interactions in the 
surrounding media (water, ice or rock). The upgoing requirement is necessary 
to reject the large background flux of downgoing cosmic ray induced muons. 
After rejecting all downward events, the fiux of atmospheric neutrinos from 
cosmic ray interactions in the earth's atmosphere remains as the background 
to a search for extra-terrestrial neutrinos. 

We can determine the sensitivity of an experiment to a fiux of extra-terrestrial 
neutrinos by optimising the model rejection potential to find the average limit 
that would be placed on that fiux if no true signal were present, and only the 
expected background atmospheric neutrino events observed. In the following 
sections, we use the searches for diffuse fluxes and point sources of extra- 
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terrestrial neutrinos to illustrate the model rejection potential technique. For 
the diffuse ffux case, the model rejection potential optimisation is done in 
one cut dimension, whereas in the point source case, we show how a two- 
dimensional optimisation is made. The method could in principle be gener- 
alised to an optimisation in n dimensions, where the point in the ra- dimensional 
cut parameter space is found where the model rejection factor is minimised. 

Predicting the rate of upward moving neutrino induced muons in 
the vicinity of an underwater or underice detector is straightfor- 
ward [8,9,10,11,12,13,14,15,16] and so we describe here only the basic ele- 
ments of the calculation. The current calculation follows a previously described 
method [13,14], where the neutrino cross sections are calculated using the 
"G" parton distribution set of Martin, Roberts and Stirling [17], the Lipari- 
Stanev [18] muon propagation code is used to track the muons to the detector, 
and the structure of the earth is accounted for using the Preliminary Refer- 
ence Earth Model [19]. An important difference between this and other work 
is the exact treatment of neutral current regeneration of neutrinos in the earth 
using a recursive importance sampling interaction algorithm [13,14]. Neutrino 
and anti-neutrino events were generated using an spectrum, and then re- 
weighted to various astrophysical flux predictions. A constant detector muon 
effective area of Ikm^ was assumed for all angles and energies.^ 



Model rejection potential optimisation for diffuse flux searches 

We consider three types of diffuse neutrino flux predictions, firstly E""^ spectra, 
an example of which is the Waxman-Bahcall diffuse upper bound [21] on the 
flux of astrophysical neutrinos. They calculate that the diffuse flux of neutrinos 
should not exceed a level 

^^*WBUB = 3.0 X 10"^ GeV cm'^ sr"^ (7) 

Mannheim, Protheroe and Rachen [22] question the assumptions made in the 
Waxman-Bahcall upper bound calculation and have computed fluxes of diffuse 
neutrinos that violate this upper bound. One of the Mannheim, Protheroe 
and Rachen models is also tested here. Finally, we determine the sensitiv- 
ity of a kilometre scale detector to the original Stecker, Done, Salamon and 



^ The results of a complete simulation of a realistic detector will be forthcoming 
from the Icecube collaboration[20]. 
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Sommers [23] diffuse neutrino ffux prediction. The atmospheric neutrino back- 
ground to these searches is taken from the calculation of Lipari [24]. In cal- 
culating this background, we have not considered the uncertain contribution 
from the prompt charm decay channel [25,26]. Including this neutrino flux in 
a background estimat^ requires incorporating the systematics into the limit 
calculation[27,28]). Or, one could consider this flux as a source on which limits 
can be placed, though difficult to separate from an extra-terrestrial source [14]. 

We consider in detail the optimisation of the model rejection potential for a 
diffuse E'"^ source of astrophysical neutrinos. We will assume that the muon 
energy at the detector can be measured, and use this as the cut against the 
atmospheric neutrino induced background events. Figure 2 shows the differ- 
ential energy distributions for the atmospheric neutrino-induced muons, and 
for the muons resulting from a diffuse neutrino source of strength equal 
to the Waxman-Bahcall upper bound. 

Figure 3 shows the integrated event rates above each muon energy. For each 
value of the expected atmospheric neutrino background, we can calculate the 
average upper limit that would be obtained by the experimental ensemble 
with no true signal, and this curve is added to the plot as a solid line. 

As the background decreases the average upper limit reduces toward 2.44, the 
value of [i^oinohs^nh) for ^6 = and Uobs = 0. Clearly, if = 0, the only 
non-zero term in the sum in equation 4 is /i9o(0, 0) = 2.44 which occurs with 
probability one (if the background is zero, then the only possible observation 
in the absence of signal is Uobs = 0). The best cut corresponds to the muon 
energy above which the ratio of the average upper limit to the expected diffuse 
signal is minimised. This is shown in figure 4. The best cut lies close to a muon 
energy of 10^ GeV, where 2.64 background and 26.69 signal events remain. The 
average upper limit for this expected background is 4.26, leading to a model 
rejection factor for the assumed fiux strength of 4.26/26.69 = 0.16, and thus 
a limit on such an E'"^ fiux of 

^^$90 < 4.8 X 10"^ GeV cm"^ s"^ sr"^ (8) 

This level of sensitivity is nearly an order of magnitude below that of the 
Waxman-Bahcall bound. 



^ For point source searches, the background is measured off source, in which case 
an uncertain theoretical prediction is not important. 
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2 3 4 5 6 7 8 

loglO(true muon energy at detector/GeV) 



Fig. 2. Differential energy distribution of true muon energies in a neutrino de- 
tector for the atmospheric neutrino flux and an E~'^ diffuse neutrino flux, of level 
£"^$90 = 3 X 10~^ GeV cm~^ s~^ sr~^, corresponding to the Waxman-Bahcall upper 
bound calculation. 

Note that this procedure is independent of the assumed signal strength, de- 
pending only the shape of the signal spectrum. 

This limit is an ideal one where we have assumed we can measure the muon 
energy exactly. A realistic detector will have a finite energy resolution, and 
we incorporate this by assuming a gaussian resolution in the logarithm of the 
true muon energy. Table 1 shows a summary of the model rejection potential 
optimisations for measurements of the true muon energy, for several assumed 
detector muon energy resolutions, and for measurements of the true neutrino 
energy, which would indicate the best possible limit that the detector could 
achieve. Also, we show what constraints would be placed on other models of 
neutrino production, using as examples the calculation of Stecker, Done, Sala- 
mon and Sommers [23] (SDSS), and one of the diffuse bounds of Mannheim, 
Protheroe and Rachen [22] (MPR). A logarithmic muon energy resolution of 
30% produces an expected limit below the Waxman-Bahcall upper bound, and 
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Fig. 3. Integrated energy distribution of true muon energies for the fluxes shown in 
figure 2. The average upper Umit that would be obtained by an ensemble of experi- 
ments with no true signal and only the expected atmospheric neutrino background 
is shown by the solid line. If the background exceeds 700, we do not calculate an 
average upper limit, thus the cut off in the curve at log;^o(-E'^) = 3.87. 

is sufficient to place strong constraints on both the SDSS and MPR models. 

The effects of increasing the detector exposure time are shown in table 2, where 
three years of livetime exposure is assumed. In each case the optimisation 
moves toward higher energy cut values, and the improvement in the expected 
limits goes approximately as the square root of the exposure time. The detector 
performance for three years exposure is also summarised in figure 5. 



^.^ Model rejection •potential optimisation for point source searches 



In the search for a point source, we can reduce the background to the search by 
cutting on both the energy of the events, and also on the angular acceptance 
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Fig. 4. Model rejection factor, fi/ug, for a diffuse flux search in a kilometre effective 
area neutrino telescope, as a function of the muon energy cut. The best average 
limit on the source flux occurs where the MRF is minimised, here at a muon energy 
just below 10^ GeV. 



window about the known direction of the source. We apply the model rejection 
potential procedure in two dimensions, producing a two dimensional surface 
of model rejection factor, jj./s, in the energy and angular acceptance (A^ 
degrees about the known direction) cut variables. As a single illustration of 
this technique, a contour plot representation of this surface for the case of an 
point source, assuming an angular resolution with Gaussian a = 3.0°, 
and an exact measure of the muon energy in the detector, is shown in figure 6. 
The best constraint on the point source (shown as a cross) is found by 
accepting events with A^' < 3.8° and log 10(£'^/GeV) > 4.5. The remaining 
background of 1.4 events leads to an average event upper limit of 3.56 and an 
optimised flux limit of 



E'^QO < 9.4 X 10-'" GeV cm"^ s 



(9) 
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Tabic 1 

Model rejection potential resTilts for various astrophysical neutrino spectra, for one 
year live time in a kilometre scale neutrino detector. The Waxman-Bahcall flux 
limits are in units GeV cm~^ sr~^. 





cut 


above cut 


Flux 


var.(log err.) 


lOffin E 






h^\"'0 J 


Flux limit (= $ x -^^) 




E„(0%) 


5.68 


51.12 


1.25 


3.45 


2.03 . 10~^ 






4.96 


26.69 


2.64 


4.26 


4.79 . 10~^ 


WBUB 




5.32 


18.77 


4.12 


4.87 


7.79 . 10~^ 




E„ (20%) 


5.68 


18.47 


22.19 


9.37 


1.52 . 10"^ 






6.40 


14.09 


32.15 


10.98 


2.35 . 10~^ 




E,{0%) 


5.86 


281.40 


0.47 


2.83 


$MPR X 1.01 . 10-2 




E^{0%) 


5.14 


127.58 


1.01 


3.29 


$MPR X 2.58 . 10^2 


MPR 


E^{10%) 


5.50 


90.10 


1.99 


3.94 


^>MPR X 4.37 . 10^2 




Ef,{20%) 


6.22 


54.55 


4.51 


5.01 


$MPR X 9.18 . 10^2 




£^^(30%) 


6.94 


40.53 


9.09 


6.51 


^MPR X 0.161 




^.(0%) 


5.68 


3347.49 


1.25 


3.45 


^SDSS X 1.03 . 10-3 






5.14 


1222.01 


1.01 


3.29 


^SDSS X 2.69 . 10-3 


SDSS 


E^{10%) 


5.32 


1039.62 


4.12 


4.87 


<5sDSS X 4.69 . 10-3 




E^{20%) 


6.04 


606.97 


7.40 


6.01 


^SDSS X 9.91 . 10-3 




Ef,{30%) 


6.76 


452.29 


14.07 


7.77 


$SDSS X 1.72 . 10-2 



The change in model rejection factor moving between adjacent contours 
corresponds to a change in the average flux upper hmit in hnear steps of 
IQ-^'^ GeV cm-2 s-^ . Thus the minimum hes in a fairly broad range in the 
two dimensional cut parameter space. 



If we assume a 30% logarithmic muon energy resolution, the optimal cuts 
change to A* < 2.5° and log 10(£;^/GeV) > 3.25, and the hmit is only a 
factor of 2.5 weaker than for the true energy measurement case. Contrast this 
to the diffuse i?-^ limit case, where the 30% energy resolution produced a 
limit a factor of 4.9 weaker, due to sole reliance of energy as the cut. In the 
point source case, much of the background rejection comes from the angular 
acceptance cut, and the energy measurement error has less effect. 
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Tabic 2 

Model rejection potential results for various astrophysical neutrino spectra, for three 
years live time in a kilometre scale neutrino detector. The Waxman-Bahcall flux 
limits are in units GeV cm~^ s"^ sr~^. 





cut 


above cut 


Flux 
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Flux limit (= $ x 


JL) 
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5.86 
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1.40 
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8.99 . 10~ 
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E^{10%) 


5.68 
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4.35 
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. 10" 
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^^(20%) 


6.58 




112.87 


4.33 


4.94 


^>MPR X 4.38 


. 10" 


-2 




£^^(30%) 


7.12 
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18.42 


8.65 


$MPR X 8.32 


. 10" 
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E,{0%) 


5.86 




8076.81 


1.40 


3.55 


^SDSS X 4.40 


. 10- 


-4 




E^{Q%) 


5.32 




2841.39 


1.06 


3.32 


^SDSS X 1.17 


. 10" 
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6.40 
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7.12 




967.18 
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^.3 Model rejection potential optimisation for gamma-ray hurst searches 

The mechanisms that power gamma-ray bursts are still not determined. If 
hadronic acceleration mechanisms are producing gamma-rays via proton in- 
teractions and subsequent neutral pion decay, then there should also be fluxes 
of high energy neutrinos. Large neutrino detectors will look for these neutrinos 
in coincidence with the satellite observations of the gamma-rays. Waxman and 
Bahcall [21] compute the expected neutrino flux by assuming that the gamma- 
ray bursts are also the source of the observed cosmic rays. 
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logio(E^) [GeV] 

Fig. 5. Summary of the limit setting potential of a kilometre-scale neutrino detec- 
tor after three years of operation. For each of the diffuse flux models under con- 
sideration, the highest curves show the predicted neutrino flux. The shaded bands 
correspond to the expected range of the limit for different assumptions of the muon 
energy resolution of the detector, the upper edge of the band corresponding to 30% 
logarithmic muon energy resolution, and the lower edge, perfect muon energy res- 
olution. The lowest curves for each flux show the expected limits if the neutrino 
energy could be measured directly. The atmospheric neutrino background is also 
shown as a band representing the flux variation from horizontal (top of band) to 
vertical (bottom of band) arrival directions. 

Just as in the point source search we optimise both the angular acceptance 
about the known source direction and the true muon energy. The additional 
cut on a time window about the known GRB time greatly reduces the back- 
ground to the search. We assume that we will test a model in which the 
neutrinos are produced within 10 seconds of the burst time. The accumulated 
number of coincident bursts is used as a measure of the detector exposure [29] . 

The model rejection factor surface after 500 sateUite-coincident bursts are seen 
in the detector's field of view is shown in figure 7. The application of a 10 sec- 
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Fig. 6. Surface of model rejection factor, Jl/ns-, for a point source search in a 
kilometre effective area neutrino telescope, where the angular acceptance about 
the assumed source direction, and the true detector muon energy are used as cut 
parameters. Adjacent contours correspond to a change in the flux upper limit of 
10"^°, with the minimum value of 9.4 x 10 GeV cm ^ s ^ indicated by the 
cross. 

ond time window cut about each of the known burst times reduces the total 
background to essentially zero and thus only loose cuts are needed - an angu- 
lar acceptance cut of < 7.0° and an energy cut of log 10(£'^/GeV) > 1.18, 
both of which arc much weaker than for the point source optimisation of sec- 
tion 4.2, are found to be optimal. Using these cuts, the remaining atmospheric 
background (0.17 events) and average upper limit (2.58) lead to an optimal 
upper hmit corresponding to a flux of 0.18 times that of the Waxman-Bahcall 
prediction. Again, the minimum lies in a broad region - the change in model 
rejection factor between adjacent contours being 6.7 X 10"^ Since the back- 
ground is so close to zero, the limit will tend to scale linearly with an increase 
in the number of observed bursts. 

Incorporating a realistic 30% logarithmic muon energy resolution leads to a 
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model rejection factor of 0.3, again only a factor of 1.7 worse than the ideal 
muon energy measurement case. The cuts stay close to those of the ideal 
case, with A^' of 7.5°, and the energy cut loosened to accept all simulated 
events (log 10(£'^/GeV) > 1.0) leading to an expected background of 0.12 and 
average event upper limit of 2.54. Due to the addition of the time cut, the 
energy measurement error has even less weakening effect on the optimal limit 
than in the point source case. 




log(true muon energy at detector/GeV) 

Fig. 7. Surface of model rejection factor, ///n^, for a Waxman-Bahcall model 
gamma-ray burst source search in a kilometre effective area neutrino telescope, 
where, in addition to angle and energy cuts, a time window cut of 10 seconds about 
each of the assumed 500 known GRB emission times is applied. The point of the 
minimum model rejection factor of 0.18 is indicated by the cross. 

5 Conclusions 

In this paper we have described a method of choosing experimental cuts in 
order to maximise the chance of placing the strongest possible constraint on 
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an expected signal model. Choosing the experimental cuts to optimise the 
model rejection factor (ratio of expected average upper limit to expected sig- 
nal) is shown to yield the best average constraint on the signal model. We 
have demonstrated this method by determining the sensitivity of kilometre- 
scale neutrino detectors to diffuse and point sources of astrophysical neutrinos, 
and find that such detectors will strongly constrain present models. We sug- 
gest that the optimised average flux upper limits described here be used as a 
standard method of comparing the capabilities of different neutrino detectors. 
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